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Abstract. Propositional Projection Temporal Logic (PPTL) is a useful 
formalism for reasoning about period of time in hardware and software 
systems and can handle both sequential and parallel compositions. In this 
paper, based on discrete time Markov chains, we investigate the proba- 
bilistic model checking approach for PPTL towards verifying arbitrary 
linear-time properties. We first define a normal form graph, denoted by 
NFGinf , to capture the infinite paths of PPTL formulas. Then we present 
an algorithm to generate the NFGinf ■ Since discrete-time Markov chains 
are the deterministic probabilistic models, we further give an algorithm 
to determinize and minimize the nondeterministic NFGinf following the 
Safra's construction. 

Keywords: projection temporal logic, probabilistic model checking, Markov 
chains, normal form graph. 



1 Introduction 

Traditional model checking techniques focus on a systematic check of the valid- 
ity of a temporal logic formula on a precise mathematical model. The answer 
to the model checking question is either true or false. Although this classic ap- 
proach is enough to specify and verify boolean temporal properties, it does not 
allow to reason about stochastic nature of systems. In real-life systems, there 
are many phenomena that can only be modeled by considering their stochas- 
tic characteristics. For this purpose, probabilistic model checking is proposed 
as a formal verification technique for the analysis of stochastic systems. In or- 
der to model random phenomena, discrete-time Markov chains, continuous-time 
Markov chains and Markov decision processes are widely used in probabilistic 
model checking. 

Linear-time property is a set of infinite paths. We can use linear-time tempo- 
ral logic (LTL) to express w-regular properties. Given a finite Markov chain M 
and an w-regular property Q, the probabilistic model checking problem for LTL 
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is to compute the probability of accepting runs in the product Markov chain M 
and a deterministic Rabin automata (DRA) for [BJ. 

Among linear-time temporal logics, there exists a number of choppy log- 
ics that are based on chop (;) operators. Interval Temporal Logic (ITL) [3] is 
one kind of choppy logics, in which temporal operators such as chop, next and 
projection are defined. Within the ITL developments, Duan, Koutny and Holt, 
by introducing a new projection construct (pi,.. . ,p m ) prj q, generalize ITL to 
infinite time intervals. The new interval-based temporal logic is called Projec- 
tion Temporal Logic (PTL) [T^]. PTL is a useful formalism for reasoning about 
period of time for hardware and software systems. It can handle both sequen- 
tial and parallel compositions, and offer useful and practical proof techniques 
for verifying concurrent systems |14I12) . Compared with LTL, PTL can describe 
more linear-time properties. In this paper, we investigate the probabilistic model 
checking on Prepositional PTL (PPTL). 

There are a number of reasons for being interested in projection temporal 
logic language. One is that projection temporal logic can express various impera- 
tive programming constructs (e.g. while- loop) and has executable subset [10111] . 
In addition, the expressiveness of projection temporal logic is more powerful 
than the classic point-based temporal logics such as LTL since the temporal 
logics with chop star (*) and projection operators are equivalent to w-regular 
languages, but LTL cannot express all w-regular properties [5]. Furthermore, the 
key construct used in PTL is the new projection operator (px, . . . ,p m ) P r j Q 
that can be thought of as a combination of the parallel and the projection op- 
erators in ITL. By means of the projection construct, one can define fine- and 
coarse-grained concurrent behaviors in a flexible and readable way. In particular, 
the sequence of processes p±, . . . , p m and process q may terminate at different 
time points. 

In the previous work [10|11|12] . we have presented a normal form for any 
PPTL formula. Based on the normal form, we can construct a semantically 
equivalent graph, called normal form graph (NFG). An infinite (finite) interval 
that satisfies a PPTL formula will correspond to an infinite (finite) path in NFG. 
Different from Buchi automata, NFG is exactly the model of a PPTL formula. 
For any unsatisfiablc PPTL formula, NFG will be reduced to a false node at the 
end of the construction. NFG consists of both finite and infinite paths. But for 
concurrent stochastic systems, here we only consider infinite cases. Therefore, we 
define NFGi n f to denote an NFG only with infinite paths. To capture the accu- 
rate semantics for PPTL formulas with infinite intervals, we adopt Rabin accep- 
tance condition as accepting states in NFGi n f. In addition, since Markov chain 
M is a deterministic probabilistic model, in order to guarantee that the product 
of M ® NFGi n f is also a Markov chain, we give an algorithm for deterministic 
NFGi n f, in the spirit of Safra's construction for deterministic Buchi-automata. 

To make this idea clear, we now consider a simple example shown in Figure 
[1] The definitions of NFGs and Markov chains are formalized in the subsequent 
sections. Let p ; q be a chop formula in PPTL, where p and q are atomic propo- 
sitions. NFGinf of p ; q is constructed in Figure HJa), where nodes v , v± and i>2 



are temporal formulas, and edges are state formulas (without temporal opera- 
tors) . vq is an initial node, vi is an acceptance node recurring for infinitely many 
times, whereas v% appears finitely many times. Figure [ljb) presents a Markov 
chain with initial state s. Let path path = (s,Si,S3). We can see that path 
satisfies p ; q with probability 0.6. Based on the product of Markov chain and 
NFGi n f, we can compute the whole probability that the Markov chain satisfies 

p ; q- 



v : p;q 

v%: true: q P 




(a) NFGi n f olp;q. (b) An Example of Markov chains. 



Fig. 1. A Simple Example for Probabilistic Model Checking on PPTL. 



Compared with Buchi automata, NFGs have the following advantages that 
are more suitable for verification for interval-based temporal logics. 

(i) NFGs are beneficial for unified verification approaches based on the same 
formal notation. NFGs can not only be regarded as models of specification lan- 
guage PTL, but also as models of Modeling Simulation and Verification Language 
(MSVL) jlOlllj . which is an executable subset of PTL. Thus, programs and their 
properties can be written in the same language, which avoids the transformation 
between different notations. 

(ii) NFGs can accept both finite words and infinite words. But Buchi automata 
can only accept infinite words. Further, temporal operators chop (p ; q), chop 
star (p*), and projection can be readily transformed to NFGs. 

(iii) NFGs and PPTL formulas are semantically equivalent. That is, every path 
in NFGs corresponds to a model of PPTL formula. If some formula is false, then 
its NFG will be a false node. Thus, satisfiability in PPTL formulas can be re- 
duced to NFGs construction. But for any LTL formula, the satisfiability problem 
needs to check the emptiness problem of Buchi automata. 

The paper is organized as follows. Section 2 introduces PPTL briefly. Sec- 
tion 3 presents the (discrete time) Markov chains. In Section 4, the probabilis- 
tic model checking approach for PPTL is investigated. Finally, conclusions are 
drawn in Section 5. 



2 Propositional Projection Temporal Logic 

The underlying logic we use is Propositional Projection Temporal Logic (PPTL). 
It is a variation of Propositional Interval Temporal Logic (PITL). 

Definition 1 Let AP be a finite set of atomic propositions. PPTL formulas 
over AP can be defined as follows: 

Q ■■= * I I OQ I Qi a Q 2 1 (Qi, ■ • • , Q m ) prj Q | Q+ 

where it € AP, Q,Qi, . . . ,Q n are PPTL formulas, O (next), prj (projection) 
and + (plus) are basic temporal operators. 

A formula is called a state formula if it does not contain any temporal op- 
erators, i.e., next (0)> projection (prj ) and chop-plus ( + ); otherwise it is a 
temporal formula. 

An interval a = (sq, Si , . . .) is a non-empty sequence of states, where Sj (i > 0) 
is a state mapping from AP to B = {true, false}. The length, |cr|, of a is uj if cr is 
infinite, and the number of states minus 1 if a is finite. To have a uniform notation 
for both finite and infinite intervals, we will use extended integers as indices. 
That is, for set No of non-negative integer and uj, we define N u = No U {to}, and 
extend the comparison operators: =, <, <, to N u by considering uj — uj, and for 
all i £ No,i < uj. Moreover, we define ^ as < — {(uj,uj)}. 

To define the semantics of the projection construct we need an auxiliary 
operator. Let a = (so,Si,...) be an interval and r±,...,rh be integers (h > I) 
such that < n < . . . < Th ^ ] cr| . 

ctJ, (ri,...,r h ) d = (s tl ,s t2 ,...,s ti ) 

The projection of cr onto r\, . . . ,r^ is the interval (called projected interval) 
where t±, . . . , ti are obtained from n, . . . , r/j by deleting all duplicates. In other 
words, ti, . . . ,ti is the longest strictly increasing subsequence of rj., . . . , Vh- For 
example, (so, si, S2, S3) 4- (0; 2, 2, 2, 3) = (sq, S2, S3). As depicted in Figure [51 the 
projected interval (so,S2,S3) can be obtained by using J. operator to take the 
endpoints of each process e, len(2), e, e, len(l). 



the projected interval 

so A s 2 S3 
• • • 

]^ /eM2 I ^-1 

■ Sq Si ^2 : S3 

X% le*2 " (e«l- 



Fig. 2. A projected interval. 



An interpretation for a PPTL formula is a tuple I = (a, i, k, j), where a is an 
interval, i, k are integers, and j an integer or uj such that i <k < j. Intuitively, 



(er, i, k, j) means that a formula is interpreted over a subinterval with the 

current state being Sfc. The satisfaction relation (|=) between interpretation 2 
and formula Q is inductively defined as follows. 

1. 2 \= n iff Sfc[7r] = irwe 

2. X |= -nQ iff IJ^ Q 

3. 2 h Qi A Q 2 iff 2 h Qi and 2 \= Q 2 

4. X |= OQ iff fc < j and (cr,i,fc+l,j) |= Q 

5. I \= {Qx , . . . , Q m ) prj Q iff there are k = ro < r% < . . . < r m ^ j such that 
(a, i, r ,rx) \= Qi and (cr, rj_i, n) |= for all 1 < I < m and (a', 0,0, |<r'|) 
|= Q for a' given by : 

(a) r m < j and a' = a \ (r , . . . ,r m ) • <T( rm+1) ..^ 

(&) r m = j and a' = a ], (ro, ■ ■ • , rh) for some < h < m. 

6. I \= Q + iff there are finitely many ro, . . . ,r n and k = ro < r\ < ... < 
r n ~i ^ r n = j (n > 1) 

such that (a,i,ro,r\) \= Q and (<r, rj-i, rj-i, r;) |= Q for all 1 < I < n or 
j = lj and there are infinitely many integers k = ro < ri < r 2 < . . . such 
that lim rt = oj and (cr, i, ro, ri) ^= Q and for Z > 1, (a, r;) |= Q. 

z— >oo 

A PPTL formula Q is satisfied by an interval a, denoted by a \= Q, if 
(cr, 0,0, |ct|) |= Q. A formula Q is called satisfiable, if a \= Q. A formula Q is 
valid, denoted by |= Q, if cr |= Q for all cr. Sometimes, we denote |= p c/ (resp. 
|= p — s> c/) by p w q (resp.^ ) and |= U(p o q) (resp. |= D(p — ?> c/)) by p = c/ 
(resp. p D q), The former is called weafc equivalence (resp. weak implication) and 
the latter strong equivalence (resp. strong implication). 

Figure [3] below shows us some useful formulas derived from elementary PTL 
formulas, e represents the final state and more specifies that the current state is 
a non-final state; OP (namely sometimes P) means that P holds eventually in 
the future including the current state; OP (namely always P) represents that P 
holds always in the future from now on; Q P (weak next) tells us that either the 
current state is the final one or P holds at the next state of the present interval; 
Prj (Pi, . . . , P rn ) represents a sequential computation of Pi,...,P m since the 
projected interval is a singleton; and P ; Q (P chop Q) represents a computation 
of P followed by Q, and the intervals for P and Q share a common state. That 
is, P holds from now until some point in future and from that time point Q 
holds. Note that P ; Q is a strong chop which always requires that P be true 
on some finite subinterval. len(n) specifies the distance n from the current state 
to the final state of an interval; skip means that the length of the interval is one 
unit of time. fin(P) is true as long as P is true at the final state while keep(P) 
is true if P is true at every state but the final one. The formula halt(P) holds if 
and only if formula P is true at the final state. 

An Application of Projection Construct 

Example 1 We present a simple application of projection construct about a 
pulse generator for variable x which can assume two values: (low) and 1 (high). 
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Fig. 3. Derived PPTL formulas. 



We first define two types of processes: The first one is hold(i) which is exe- 
cuted over an interval of length i and ensures that the value of x remains constant 
in all but the final state, 

hold(i) d = frame (i) A len(i) 

The other is switch(j) which is ensures that the value of x is first set to 
and then changed at every subsequent state, 

switch(j) = x = A len(j) A □(more — > Q)x = 1 — a?) 

Having defined hold{i) and switch(j), we can define the pulse generators 
with varying numbers and length of low and high intervals for x, 

pulse(i\, . . . , ik) = (hold(ii), . . . , hold{iu)) prj switch(k) 
For instance, a pulse generator 

pulse(3, 5, 3, 4) ^ f (hold(3), hold(5), hold(3), hold{A)) prj switch(A) 

can be shown in Figure 0J 

Let Q be a PPTL formula and Q p £ AP be a set of atomic propositions in 
Q. Normal form of PPTL formulas can be defined as follows. 

Definition 2 A PPTL formula Q is in normal form if 

no n 

Q=(\J Q ej Ae)V(\/Q c ,AOQ/,) 
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Fig. 4. A Pulse Generator 



mo m 

where Q ej = A Qjk,Qc z = A 9ih, \Qp\ = h 1 < ™>o < I, 1 < m < I; q jk ,qih € 

k=l h=l 

Q p , for any r G Q p , r means r or ^r; Qfi is a general PPTL formula. For 

no n 

convenience, we often write Q e A e instead of V Q ej A e and \J Qi A OQi 

j=Q i=0 

n 

instead of V Qc, A OQf v Thus, 

4=0 

n 

Q = (Q e Ae) V(VQi AOQO 

4 = 

where Q e and Qi are state formulas. 

Theorem 1 For any PPTL formula Q, there is a normal form Q' such that 
Q = Q'. M 

3 Probabilistic System 

We model probabilistic system by ( discrete-time) Markov chains (DTMC). With- 
out loss of generality, we assume that a DTMC has a unique initial state. 

Definition 3 A Markov chain is a tuple M = (S, Prob, i-mit, AP, L), where S 
is a countable, nonempty set of states; Prob : S x S — > [0, 1] is the transition 
probability function such that ^2 Prob(s, s') — 1; ii n u : S — > [0, 1] is the initial 

distribution such that ^2 knit(s) = 1 ; and AP is a set of atomic propositions 

ses 

and L : S — !> 2 AP a labeling function. 

As in the standard theory of Markov processes jS], we need to formalize a 
probability space of M that can be defined as ipM — (ft, Cyl,Pr), where ft de- 
notes the set of all infinite sequences of states (sojSi, . . .) such that Prob(si, S4+1) > 



for all i < 0, Cyl is a er-algebra generated by the basic cylindric sets: 

Cyl(s , ■ ■ ■ , s n ) = {path £ Q \ path = s , si, . . . , s„, . . .} 
and Pr is a probability distribution defined by 

Pr M (Cyl(s ,...,s n )) = Prob{s , . . . , ««) 

= Prob(si, s i+ i) 

0<i<n 

If p is a path in DTMC M and Q a PPTL formula, we often write p |= Q to 
mean that a path in DTMC satisfies the given formula Q. Let path(s) be a set 
of paths in DTMC starting with state s. The probability for Q to hold in state s 
is denoted by Pr M (s \= Q), where Pr M (s \= Q) = Prf 1 {p e path{s) \ p\=Q}. 

4 Probabilistic Model Checking for PPTL 

In [12] , it is shown that any PPTL formulas can be rewritten into normal form, 
where a graphic description for normal form called Normal Form Graph (NFC) 
is presented. NFC is an important basis of decision procedure for satisfiability 
and model checking for PPTL. In this paper, the work reported depends on the 
NFC to investigate the probabilistic model checking for PPTL. 

However, there are some differences on NFC between our work and the pre- 
vious work in [10111112] , First, NFG consists of finite paths and infinite paths. 
For concurrent stochastic systems, we only consider to verify w-regular proper- 
ties. Thus, we are supposed to concern with all the infinite paths of NFG. These 
infinite paths are denoted by NFG in j. Further, to define the nodes which recur 
for finitely many times, [H] uses Labeled NFG (LNFG) to tag all the nodes in 
finite cycles with F. But it can not identify all the possible acceptance cases. As 
the standard acceptance conditions in w-automata, we adopt Rabin acceptance 
condition to precisely define the infinite paths in NFGi n f. In addition, since 
Markov chain M is a deterministic probabilistic model, in order to guarantee 
that the product of M ® NFGi n f is also a Markov chain, the NFGi n f needs 
to be deterministic. Thus, following the Safra's construction for deterministic 
automata, we design an algorithm to obtain a deterministic NFGi n j. 

4.1 Normal Form Graph 

In the following, we first give a general definition of NFG for PPTL formulas. 

Definition 4 (Normal Form Graph [10.12J) For a PPTL formula P, the 
set V(P) of nodes and the set of E(P) of edges connecting nodes in V(P) are 
inductively defined as follows. 



l.Pg V(P); 



2. For all Q g V (P) / {e , false} , if Q = (Q e Ae)V( V QiAOQ'i), then e g 

(Q,Q e ,e) g P(P); e V(P), (g,Q l ,Q-) 6 £(P) for alU, 1 < i < n. 

The NFG of PPTL formula P is the directed graph G = (V(P),E(P)). 

A finite path for formula Q in NFG is a sequence of nodes and edges from 
the root to node e. while an infinite path is an infinite sequence of nodes and 
edges originating from the root. 

Theorem 2 (Finiteness of NFG) For any PPTL formula P, \V{P)\ is finite 

Theorem [2] assures that the number of nodes in NFG is finite. Thus, each 
satisfiable formula of PPTL is satisfiable by a finite transition system (i.e., fi- 
nite NFG). Further, by the finite model property, the satisfiability of PPTL is 
decidable. In [12 , Duan etal have given a decision procedure for PPTL formulas 
based on NFG. 

To verify w-regular properties, we need to consider the infinite paths in NFG. 
By ignoring all the finite paths, we can obtain a subgraph only with infinite paths, 
denoted NFG ln j. 

Definition 5 For a PPTL formula P, the set Vj„/ (P) of nodes and the set of 
Einf(P) of edges connecting nodes in V^„/(P) are inductively defined as follows. 

1. PgF in/ (P); 

n 

2. For all Q g V inf (P), if Q = (Q e A e) V ( V ft A OQ<)» then ^ e V mf {P), 

i=0 

(Q, Q h QJ) g E mf (P) for all *, 1 < * < n. 

Thus, NFGmf is a directed graph G' — (V ln j(P),E in f(P)). Precisely, G 1 is a 
subgraph of G by deleting all the finite path from node P to node e. 

In fact, a finite path in the NFG of a formula Q corresponds to a model (i.e., 
interval) of Q. However, the result does not hold for the infinite case since not 
all of the infinite paths in NFG can be the models of Q. Note that, in an infinite 
path, there must exist some nodes which appear infinitely many times, but there 
may have other nodes that can just recur for finitely many times. To capture 
the precise semantics model of formula Q, we make use of Rabin acceptance 
condition as the constraints for nodes that must recur finitely. 

Definition 6 For a PPTL formula P, NFGi n f with Rabin acceptance condition 
is defined as Gn a Mn = {Vinf(P), E in f(P), vq,(2), where V{P) is the set of nodes 
and E(P) is the set of directed edges between V(P), vq g V(P) is the initial 
node, and Q — {(Pi, Pi), . . . , (E k ,F k )} with E i} F t g V(P) is Rabin acceptance 
condition. We say that: an infinite path is a model of the formula P if there 
exists an infinite run p on the path such that 



3{e, f) g n.{ P n e = 0) a (p n f + 0) 



Example 2 Let Q be PPTL formulas. The normal form of OQ are as follows. 



OQ = true ; Q 

= (e V Otrue) ; Q 

= (e ; Q) V (Oirue ; Q) 

= Q V 0(true;Q) 

= (Q A e) V (Q A Otrue) V O^Q 

= (Q A e) V (Q A 0(e V 0™e)) V (JOQ 

The NFG and NFGi n f with Rabin acceptance condition of OQ are depicted 
in Figure [5] By the semantics of formula OQ (see Figure [3]), that is, formula 
Q holds eventually in the future including the current state, we can know that 
node OQ must cycle for finitely many times and node T (i.e., true) for infinitely 
many times. 




where n = {(OQ, T)}. 



(i) NFG of <>0- ( a ) NFG W of 0<3- (iii) NFG lnf with Rabin acceptance condition of 00- 

Fig. 5. NFG of OQ. 



4.2 The Algorithms 

To investigate the probabilistic model checking problem for interval-based tem- 
poral logics, we use Markov chain M as stochastic models and PPTL as a spec- 
ification language. In the following, we present algorithms for the construction 
and dcterminization of NFGi n f with Rabin acceptance condition respectively. 

Construction of NFGi„f In Table [T] we present algorithm NFGi n f(Q) for 
constructing the NFGinf with Rabin acceptance condition for any PPTL for- 
mula. Algorithm NF(Q) can be found in [T2], which is used for the purpose of 
transforming formula Q into its normal form. For any formula R G Vi n f (Q) and 
visit(R) = 0, we assume that P = NF(R) is in normal form, where visit(R) = 
means that formula R has not been decomposed into its normal form. When 
P ee Vti Pi V OPI or P ee (Vj =1 Pei A e) V (\/- = i P ^ O^O, if Pi is a new 



Table 1. Algorithm for constructing NFGi n f with Rabin condition for a PPTL 
formula. 



Function NFG mf (Q) 

/*precondition: Q is a PPTL formula, NF(Q) is the normal form for Q * / 
/*postcondition: NFGinf (Q) outputs NFGinf with Rabin condition of Q, 

G Rabm = (V inf (Q),E inf (Q),vo, O) */ 



/*initialization*/ 



begin function 

V inf (Q) = {<?}; E inf (Q) = 0; visit(Q) = 0; v = Q; E = F = 
while there exists R € Pm/(<2) and visit (R) == 
do P = NF(R); visit (R) = 1; 
switch(P) 

h 

case P = V P e j A e: break; 

3 = 1 

case P = V P t A OP/ or P = ( V Pe 3 A e) V ( V P> A OP'): 

i= 1 j — 1 i — 1 

foreach i (1 < i < k) do 
if ^= false) and P/ £ 
then visit(P/) = 0; 

/*Pj is not decomposed to normal form*/ 

V m/ (Q) = l/ m/ (Q)U U{P'}; 

i=i 

*W(Q) = P m /(0) U U {(P,P,P/)}; 

i=i 

if -.(P/ = false) and P/ € V m f(Q) 

k 

then P m/ (Q) = E inf (Q) U U {(P, P, P')}; 

i=i 

when P' = R do /*selfdoop*/ 

if P is Qi ; Q 2 then E = P U {P} else P = P U {P} 

for some node P" £ Vi„/(Q); 
fc 

let NF(R") = V ft A OP or 

ft. A; 

NF(R") = ( V Pej A e) V ( V Pi A OP); 
i=i i=i 
/*nodes P and P" form a loop*/ 

when P' = P" (P" / P) do 

if P, P" P then P = P U {{P, P"}} 
else E = PU {{P,P"}}; 



break; 
end while 
return G Rabln ; 
end function 



formula (node), that is, P[ Vi n f, then by Definition^ we add the new node P[ 
to Vi n f and edge (R, Pi, P-) to E in f respectively. On the other hand, if P[ G Vinf, 
then it will be a loop. In particular, we need to consider the case of R = Q\ ; Q2- 
Because Qi ; Q2 (Qi chop Q2, defined in FigJ3]) represents a computation of Qi 
followed by Q2, and the intervals for Q\ and Q2 share a common state. That 
is, Qi holds from now until some point in future and from that time point Q 2 
holds. Note that Qi ; Q 2 used here is a strong chop which always requires that 
Qi be true on some finite subinterval. Therefore, infinite models of Qi can cause 
R to be false. To solve the problem, we employ Rabin acceptance condition to 
constraint that chop formula will not be repeated infinitely many times. 

By Theorem [21 we know that nodes V(Q) is finite in NFG. Since Vj n /(Q) C 
V(Q), so Vi n f(Q) is finite as well. This is essential since it can guarantee that 
the algorithm NFGi n f(Q) will terminate. 

Theorem 3 Algorithm NFGinf (Q) always terminates. 

Proof: Let Vi„f(Q) = {wi, . . . , v n }. When all nodes in Vi n f are transformed into 
normal form, we have visit{vi) == 1 (1 < i < n). Hence, the while loop always 
terminates. 

We denote the set of infinite paths in an NFGi n f G by path(G) = {pi, . . . ,p m }, 
where pi (1 < i < m) is an infinite path from the initial node to some acceptable 
node in F. The following theorem holds. 

Theorem 4 GnaUn and G' Rabin are equivalent if and only if path(G Rabin) = 
path(G' Rabm ). 

Let Q be a satisfiable PPTL formula. By unfolding the normal form of Q, 
there is a sequence of formulas (Q, Qi, Q' 1: Q2, Q' 2 , ■ ■ ■)• Further, by algorithm 
NFGi n f, we can obtain an equivalent NFGinf to the normal form. In fact, an 
infinite path in NFGinf of Q corresponds to a model of Q. We conclude this fact 
in Theorem [5j 

Theorem 5 A formula Q can be satisfied by infinite models if and only if there 
exists infinite paths in NFGi„f of Q with Rabin acceptance condition. 

Determinization of NFGi n f Buchi automata and NFGinf both accept u>- 
words. The former is a basis for the automata-theoretic approach for model 
checking with liner-time temporal logic, whereas the latter is the basis for the 
satisfiability and model checking of PPTL formulas. Following the thought of the 
Safra's construction for deterministic Buchi automata |15j . we can obtain a de- 
terministic NFGi n f with Rabin acceptance condition from the non-deterministic 
ones. However, different from the states in Buchi automata, each node in NFGinf 
is specified by a formula in PPTL. Thus, by eliminating the nodes that contain 
equivalent formulas, we can decrease the number of states in the resulting de- 
terministic NFGinf to some degree. 



The construction for deterministic NFGi n f is shown in Table [5J For any 
R G Vl n JQ), R is a Safra tree consisting of a set of nodes, and each node v 
is a set of formulas. By Safra's algorithm [TS], we can compute all reachable 
Safra tree R' that can be reached from R on input Pj. To obtain a deterministic 
NFGi n f , we take all pairs (P„, P„) as acceptance component, where E v consists 
of all Safra trees without a node v, and F v all Safra trees with node v marked 
'!' that denotes v will recur infinitely often. Furthermore, we can minimize the 
number of states in the resulting NFGi n f by finding equivalent nodes. Let R = 
{vq, ■ • ■ , v n } and R' — {v' , . . . , v' n } be two Safra's trees, where R, R' G V' in f , 
nodes = {Qi, Q2, ■ ■ ■} and v[ — {Q[, Q' 2 , • ■ •} be a set of formulas. For any 
nodes Uj and v[, if we have Uj = v[, then the two Safra's trees are the same. 
Moreover, we have Vi — v[ if and only if \/™ =1 Qj = Vj=i*3j- The decision 
procedure for formulas equivalence can be guaranteed by satisfiability theorems 
presented in [T!3] . 



Table 2. Algorithm for Deterministic NFGinf. 



Function DNFG(Q) 

/*precondition: G Rabln = {V lnf (Q) , E inf (Q) , v , H) is an NFG mf for PPTL formula Q. */ 
/^postcondition: DNFG(Q) outputs a deterministic NFGinf and 
G' Rabin = (VL f (Q),E' mf (Q),v' ,n') y 

begin function 

VLfiQ) = {Q}; EinfiQ) = kv'o=v -E v = F v = 0; /^initialization*/ 

while i? £ Vj' n /((5) and there exists an input Pi do 
foreach node v £ R such that R n P / 

do w' = « n P; P' = P U {«'}; /* create a new node «' such that v' is a son of v* / 
foreach node v in R' 

dov = {PI G F fa/ (Q) I 3(P,P,P') G Einf(Q), P G «}; /^update P'*/ 
foreach v G P' do if Pi G u such that Pi G left sibling of v then remove Pi in v; 
foreach v G P' do if v = then remove v; 

foreach w G P' do if ui, . . . , u n are all sons of t) such that v — Ui{ui} (1 < i < n) 
then remove Mi; mark u with !; 
(Q) = {P'} U Vi f (Q) ; P^ (Q) = (P, P , P') U P^ (Q) ; 
end while 

/*Rabin acceptance components*/ 

E v = {P G F/„y(Q) I R is Safra tree without node v}; 

F v — {R G V t ' n f(Q) I R is Safra tree with v marked !}; 
return G' Rabm ; 
end function 



4.3 Product Markov Chains 



Definition 7 Let M — (S, Prob, Limit, AP, L) be a Markov chain M, and for 
PPTL formula Q, G Ra bin = (V ln f(Q),E mf (Q),v Q , Q) be a deterministic NFG lnf , 
where Q = {(E 1 ,F 1 ), . . . , (E kl F k )}. The product M G Rabm is the Markov 
chain, which is defined as follows. 



where 



M (8) G Rab in = (Sx V in f(Q),Prob', i zmt , {acc}, V) 
L'((s,Q')) = 



{acc} if for some F i7 Q' 6 Fj, 
and Q' £ Ej for all Ej , 
1 < i,j < k 
otherwise 



LuiM')) = 

and transition probabilities are given by 



knit if (Q, L(s),Q') E E in f 
otherwise 



Prob'((s',Q'),(s",Q")) 
= f Props', «") if (Q' 5 L(s"),Q") £ 
1 otherwise 

A bottom strongly connected components (BSCCs) in M ®Gu a bin is accept- 
ing if it fulfills the acceptance condition £2 in G Ra bin- 

For some state s £ M, we need to compute the probability for the set of 
paths starting from s in M for which Q holds, that is, the value of Pr AI (s \= Q). 
From Definition it can be reduced to computing the probability of accepting 
runs in the product Markov chain M ® G Ra bi n ■ 



Theorem 6 Let M be a finite Markov chain, s a state in M, G Rabin a deter- 
ministic NFGi n f for formula Q, and let U denote all the accepting BSCCs in 
M ® G Rabin ■ Then, we have 

Pr M (s h GRaUn) = Pr M ® G ^««(( S ,Q') |= OU) 

where (Q,L(s),Q') e E mf . 

Corollary 7 All the w-regular properties specified by PPTL are measurable. 

Example 3 We now consider the example in Figure [T] Let M denote Markov 
chain in Figure Gib). The probability that sequential property p ; q holds in 
Markov chain M can be computed as follows. 

First, by the two algorithms above, deterministic NFGinf with Rabin con- 
dition for p ; q is constructed as in Figure HJa) , where the Rabin acceptance 
condition is Q — (vi, Vu)- Further, the product of the Markov chain and NFGinf 
for formula p ; q is given in Figure [5] 



0.3 



Fig. 6. The Product of Markov chain and NFGi n f in Figure [TJ 

From FigureEJ we can see that state (ss,V2) is the unique accepting BSCC. 
Therefore, we have 

Pr M {s \=G Rabin ) 
= Pr M ® G ™«((s,v 1 )\=0(s 3 ,v 3 )) 
= 1 

That is, sequential property p ; q is satisfied almost surely by the Markov chain 
M in Figure [Hb). 

5 Conclusions 

This paper presents an approach for probabilistic model checking based on 
PPTL. Both propositional LTL and PPTL can specify linear-time properties. 
However, unlike probabilistic model checking on propositional LTL, our approach 
uses NFGs, not Buchi automata, to characterize models of logic formulas. NFGs 
possess some merits that are more suitable to be employed in model checking 
for interval-based temporal logics. 

Recently, some promising formal verification techniques based on NFGs have 
been developed, such as |13|14j . In the near future, we will extend the exist- 
ing model checker for PPTL with probability, and according to the algorithms 
proposed in this paper, to verify the regular safety properties in probabilistic 
systems. 
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(a) : g terminates before ^3 
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so si s 2 s 3 s 4 s 5 
(b): P3 terminates before q 
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so si s 2 s 3 
(c): g and pz terminate at the same poi: 



